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Abstract 

The common task in matrix completion (MC) and robust principle component analysis 
(RPCA) is to recover a lo-w-rank matrix from a given data matrix. These problems gained great 
attention from various areas in applied sciences recently, especially after the publication of the 
pioneering -works of Candes et al.. One fundamental result in MC and RPCA is that nuclear 
norm based convex optimizations lead to the exact lo-w-rank matrix recovery under suitable 
conditions. In this paper, wc extend this result by sho-wing that strongly convex optimizations 
can guarantee the exact lo-w-rank matrix recovery as -well. The result in this paper not only 
provides sufhcient conditions under which the strongly convex models lead to the exact lo-w-rank 
matrix recovery, but also guides us on ho-w to choose suitable parameters in practical algorithms. 

1 Introduction 

There is a rapidly growing interest in the recovery of an unknown low-rank or approximately low- 
rank matrix from a sampling of its entries. This problem occurs in many areas of applied sciences 
ranging from machine learning [l] and control [2^ to computer vision [36], and it has regained 
great attention since the publication of the pioneering works [6l[9ll30| . Under different settings, low- 
rank matrix recovery problems can be solved by nuclear norm based convex optimizations without 
any loss of accuracy if the underlying matrix is indeed of low rank [6l-l9l l30f[33] . These fundamental 
results have had a great impact in engineering and applied sciences. Recently, they have shown their 
tremendous powers in many practical applications such as video restoration p^[20] , cognitive radio 
networks [25], decomposing background topic from keywords [25, low-rank textures capture |40] . 
and 4D computed tomography |17j . 

In order to design efficient algorithms for low-rank matrix recovery problems, instead of directly 
solving the original convex optimizations, we sometimes use their strongly convex approximations; 
see, e.g., [3l[37j. In this paper, we will show that these strongly convex programmings guarantee 
the exact low-rank matrix recovery as well. Our result not only provides sufficient conditions under 
which the strongly convex models lead to the exact low-rank matrix recovery, but also guide us on 
how to choose suitable parameters in practical algorithms. 



*Department of Mathematics and Systems Science, College of Science, National University of Defense Technology, 
Changsha, Hunan, 410073, P.R.China. Corresponding author. Email: h.zhangl984@163.com 

^Department of Mathematics, University of Iowa, Iowa City, lA 52242, USA. Email: jianfeng-cai@uiowa.edu 

■'Department of Mathematics and Systems Science, College of Science, National University of Defense Technology, 
Changsha, Hunan, 410073, RR.China. 

''Department of Mathematics and Systems Science, College of Science, National University of Defense Technology, 
Changsha, Hunan, 410073, RR.China. 



1 



1.1 Convex programming 

In this paper, we mainly focus on two specific low-rank matrix recovery problems, namely, the low- 
rank matrix completion (MC) [HlGll^ and the robust principle component analysis (RPCA) [9ll37]. 

Let M G ]R"iX"2 a,n unknown low-rank matrix. In MC, we would like to recover M from its 
partially known entries 

{rriij, E Q}, 

where O ^ [rii] x [77-2] is the set of indices of known entries. Here [n] stands for the set {1, . . . ,n}. 
Define as 

0, otherwise. 

Then the sampled data in MC can be represented as VnM. The goal of MC is to recover the 
unknown low-rank matrix M from Vq,M. Since the matrix we are seeking is of low rank, a natural 
way to solve MC is to find the lowest rank matrix among the feasible set {X G M"i^"2 . J>^x = 
PqM}. This leads to the following optimization problem 

minimize : rank(X) 
subject to : V^X = VnM. (1) 

Unfortunately, this problem is known to be NP-hard and the objective is non-convex. A popular 
alternative is to relax ([1]) to its nearest convex problem 

minimize : ||^||* 
subject to : VqX = VqM. (2) 

Here ||X||* is the summation of the singular values of X. Note that \\X\\^ is the best convex lower 
bound of the rank function on the set of matrices whose top singular value is bounded by one |15j . 
It was shown that, under suitable assumptions, the underlying low-rank matrix M can be exactly 
recovered with high probability if il. is uniformly randomly drawn from all subsets of [ni] x [712] 
with the same cardinality. 

In RPCA, the sampled data matrix, denoted by -D, is M with a small portion of its entries 
being corrupted. The goal is to restore the low-rank matrix M from its partially corrupted data 
matrix D. Since only a small portion entries are corrupted, the noise matrix D — M is a sparse 
matrix. Therefore, D consists of two components: a low-rank matrix (the underlying matrix M) 
and a sparse matrix (the noise). Thus, the RPCA problem can be solved by 

minimize : rank(L) + AUSHo 
subject to: D = L + S, (3) 

where \\S\\() stands for the number of nonzero entries of S. This problem is NP-hard and non- 
convex. Again, one can exploit the convex relaxation technique used in the MC problem. Notice 
that the convex lower bound of the zero- norm function || • ||o in the infinity-norm unit ball is the 
1-norm || • ||i (the summation of absolute values). Therefore, the convex relaxation of ^ is [9lll2j 

minimize: ||L||* + AUS"!!! 
subject to: D = L + S. (4) 



where A is a parameter balancing the low-rank and sparse components. It was shown in [9l[T2] that 
the convex optimization can recover the low-rank matrix M exactly under suitable assumptions. 



2 



1.2 Strongly convex programming 



There exist many efficient algorithms to solve the convex optimization problems ([2]) and ^ even 
when the scale of the problems is large up to 10^ x 10^, including [2l[211[26l[35llll] for ([2]) and 
[9l l22p23p37] for @. However, instead of solving the convex minimizations ^ and directly, some 
of the aforementioned algorithms solve their approximations involving strongly convex objectives. 
In particular, the singular value thresholding (SVT) algorithm [2] in MC uses Uzawa's algorithm 
to solve 

„ „ 1 „ „2 

minimize : ||^||* H II^IIf 

2r 

subject to : VqX = VnM, (5) 
and the iterative thresholding(IT) algorithm [37] in RPCA solved 

minimize : ||L||* + 77-||i|||' + AIIS*!]! + 77-||5'|||^ 
2t 2t 

subject to : D = L + S. (6) 

where r is some positive penalty parameter. One of the main advantages of using strongly convex 
programmings is that a broader range of existing optimization methods in the literature can be 
applied to the MC and RPCA problems. For example, it is well known that the convex conjugate of 
a strongly convex function is differentiable [3^. Therefore, the Lagrange dual of the strongly convex 
programmings ^ and ^ are differentiable, and hence smooth convex optimization methods can 
be applied to the dual. In fact, the SVT algorithm [2] and the IT algorithm [37] are the gradient 
algorithms applied to the Lagrange dual of ([5]) and ([6]) respectively (a.k.a. Uzawa's algorithm). 
Additionally, one can exploit Nestrov's optimal scheme for smooth optimization [28] and even 
quasi-Newton method (e.g. L-BFGS [38]), just to name a few. 

However, the convex conjugate of or \\X\\i is not differentiable so that the Lagrange dual 

of ([2]) or dH) is not smooth. That gives us a part of reason why some algorithms exploit strongly 
convex approximations to solve ([2]) and ([4]). On the other hand, it follows from [2ll37j and standard 
convex optimization theory that, when r tends to infinity, the strongly convex optimization ([5]) and 
([6|) become ([2]) and (j3]) respectively. Therefore, in order to get the exact low-rank matrix recovery, 
we have to choose an infinite r in the SVT and IT algorithms, which is impractical. Fortunately, it 
has been observed empirically in [2 ll37] that a finite r is enough for the purpose of the exact low-rank 
matrix recovery. So a natural theoretical question is whether ([5]) and ([6]) with a finite r lead to the 
exact low-rank matrix recovery in MC and RPCA. Futhermore, the empirical convergence speed 
of the SVT and IT algorithms depend on r. The smaller r is, the faster the algorithm converges. 
Therefore, it is interesting to find a lower bound of r for the exact low-rank matrix recovery. These 
two questions are answered positively in this paper. 

The related literature can be traced back to the linearized Bregman iteration (LBI) algorithm [3]- 
[5,29.38.39], which can approximately but efficiently solve the basis pursuit problem [13] min2,{||x||i : 
Ax = b} in compressed sensing [10lllipi4j . An interesting phenomenon about the LBI algorithm was 
discovered in [3tHll38] : the LBI algorithm converges to a strongly convex optimization mina;{||a;||i -|- 

^The paper [37] contains a critical error on the theoretical analysis, which had been discov- 
ered and removed by Emmanel Candes of Standford. Fortunately, the corrected version does not 
change the original convex model and has no effect on its iterative thresholding algorithm. See 
|http : //books . nips . cc/papers/f iles/nips22/MIPS2009_01 16_correction . pdf [ 
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: Ax = b} whose solution is the same as the basis pursuit problem when the parameter 
r beyond a finite scaler under some suitable conditions. In other words, under some suitable 
conditions, a convex programming is equivalent to a strongly convex programming when r is large 
enough. In |42| . this idea is extended to the MC problem. Unfortunately, the bound is very rough, 
and the method used there has many limitations and can not be extended to the RPCA problem. 
In [16], a generic exact regularization result is given, but the conditions are not easy to check for 
both MC and RPCA problems. 

1.3 Contributions and organization 

In the paper, we prove that, if r exceed some value, then under some suitable conditions strongly 
convex programming ([5]) and ([6]) can recover low-rank matrices exactly in MC and RPCA problems 
respectively. The explicit lower bounds of r are also given, and the lower bounds greatly improve 
the result in |42j . 

The significance of the paper is two-folded. Firstly, sufficient conditions under which the strongly 
convex programming leads to exact low-rank matrix recovery are derived. This, in turn, allows us 
to exploit a broader range of existing optimization method in the literature. Secondly, when we 
prefer to minimizing the strongly convex objectives for designing fast algorithms, we only need 
to set a finite parameter r beyond some value determined by the given data matrix, which lead 
to faster convergence of the algorithms compared with the one using r close to infinity. In other 
words, it provides some guidance on how to choose suitable parameters in practical algorithms. 

The remainder of the paper is organized as follows. In Section 2, we provide a brief summary 
of preliminaries including notations, assumptions, and some existing results. In Section 3, we show 
that strongly convex programming ([5]) and ([6]) lead to the exact low rank matrix recovery, and we 
give explicit expressions of the lower bounds of r. Conclusion and further works are discussed in 
Section 4. 



2 Preliminaries 

In this section, we give some notations and existing results that will be used later in this paper. 
2.1 Notations 

Let X, y G ]^"'ix"'2 matrices. The Euchdean inner product in M"iX"2 defined as {X, Y) : = 

trace(X*y), where X* stands for the transpose X. Then the Euclidean norm of X, denoted by 
is II-'^^IIf '■= \/ {X,X), and it is also known as the Probenius norm. We denote the i-th 
nonzero singular value of matrix X by ai{X), where 1 < i < rank(X). 

The Forbenius norm also equals to the Euclidean norm of the vector of singular values, i.e., 

1 /2 

W^Wf = {Y.i^K^)) ■ The spectral norm of X, denoted by ||X||, is ||X|| := ai{X), the largest 
singular value of X. The nuclear norm of X is the summation of its singular values, i.e. \\X\\^ := 
^i<yi{X). The maximum entry of X (in absolute value) is denoted by ||^||oo '■= niaxjj \Xij\. The 
li norm of a matrix viewed as a long vector is denoted by ||X||i := Yli j 
It can be easily verified that 

f ll^lloo < II^IIf < ll^lli < nin2||X|| oo, 

I ll^ll < II^IIf < ll^ll* < Vrankjx) • \\X\\f < rank(X) • ||X||. ^ ^ 
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The dual relationship is also important in our derivation. More precisely, we have 



ll^ll = sup||y||,<i(x,y), ||x||* = sup||y|j<i(x,y), 

||X||i = SUp||y||^<l(X,y), ||X||oo = SUP||y||^<i(X,y). 

It follows directly that \{X,Y)\ < \\X\\ ■ \\Y\\^ and \{X,Y)\ < \\X\\i ■ \\Y\\oo- 

We denote M E ]K'"i^'"2 |-,g ^y^q unknown low-rank matrix that is to be recovered in the MC 
and RPCA problems. Recall that the available data in the MC problem is VnM, where $7 is the 
index set of known entries. In the RPCA problem, the available data is D. We will denote the 
sparse component, or the noise, by 5o = -D — M. Without ambiguity, Q is used as the support of 
5*0 in the RPCA problem. 

Let n = max{ni,n2}. We will always use r := rank(M), and denote the singular value decom- 
position of M by 

r 
i=l 

where ai, . . . , o"r are the positive singular values of M, and U = [ui, . . . ,Ur] and V = [vi, . . . ,Vr] 
are the matrices of the left- and right-singular vectors. 
Let T be the linear subspace of M"i^"2 

T := {UXl+X2V*,Xi G M^^xr^j^^ ^ M"ix^}, (9) 

and T"*" be its orthogonal completion. Define Vt and "P^x be the orthogonal projections onto T 
and T"*- respectively. Let Vu and Vy represent the projection operators onto the spaces spanned 
by the columns of U and V respectively, i.e. Vu = UU* and Vy = VV* . The operator can be 
written as 

r^Y = ruY + Yrv -VvYVv- (lO) 

To save notations, $7 also denotes the linear space of the matrices supported on Q, so that the 
projection onto Q is Vn in ([1]). 

It is well known that any subgradient [34] of the nuclear norm function at M is of the form 



UV* + W, (11) 

where W S IR'^i^^'^a matrix satisfying 

r^w = o, liw^ll < 1. (12) 

The subgradient of the li norm function at Sq is 

sgn(5o) + F, (13) 

where F satisfies VqF = and ||-F||oo < Ij and sgn(S'o) is the matrix whose entries are the signs of 
those of 5*0. 

Finally, we shall also manipulate some linear operators which act on the space M"^^"^^ ■\Ye will 
use calligraphic letters for these operators as in A{X). In particular, I : M"-iX"-2 _^ ]gnixn2 denotes 
the identity operator and I ^ A means that ^— X is symmetric positive semidefinite, and of course 
I ~< A means that ^ — X is symmetric positive definite. We introduce the operator norm, denoted 
by ll^ll and defined as ||.4|| = sup|||j!(^||^<ii, H^XHi?. 
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2.2 Existing results 

In this subsection, we present some existing results for exact low-rank matrix recovery via convex 
optimization. For this, we need some assumptions on the unknown low-rank matrix M and the set 

Let us first give assumptions on M. It is observed in [6l[8] that, for the MC problem, it is 
impossible to recover a matrix which is equal to zero in nearly all of its entries unless all of the 
matrix entries are observed. For the RPCA problem, the authors in [9lll2j observed if the low-rank 
component is also sparse, then it is impossible to decide whether it is low-rank or sparse. These 
observations lead to the introduction of the incoherence assumptions that will be introduced in the 
following. There are some other conditions to characterize the low-rank sparse decomposition, e.g., 
the uncertainty principle and rank-sparsity incoherence in |12| . 



Assumption 1 (Incoherence Assumption). We say that the matrix M obeys the incoherent as- 
sumption with parameter fi, if the following statements are valid: 

max\\U*e^^f < ^, max\\V*eff < ^ (14) 



and 



\UV*\U < (15) 
nin2 



where e[^^ and e\^ are the i-th vector in the canonical basis o/M"^ andW'^ respectively. 

A stronger assumption, namely the strong incoherent condition, can also guarantee exact matrix 
completion via convex programming [7]- It has been shown that the strong assumption implies the 
weaker one in [l3], but for the easy presentation of the existing results, we shall introduce both of 
them. 

Assumption 2 (Strong Incoherence Assumption). We say that the matrix M obeys the strong 
incoherent assumption with parameter /Ui, if for any /i < /^i, the inequality U5\) holds true and the 
following statements are valid: for all pairs {a, a) € [ni] x [ni] and {b,b) € [712] x [^2], it holds that 



^'^^'\vV*ef)-i-^l,^,,\<^^^. 



Next we move to the assumptions on il., the sampling pattern in the MC problem and the 
sparse pattern of the sparse matrix in RPCA. The theorems stated in [6l|9] always assume that 
is uniformly randomly drawn from the set {A : |A| = m, A C [rii] x [n2]}, where | • | stands for the 
cardinality and m = However, as pointed out in [9], it is a little more convenient to work with 
the Bernoulli model Q = {{i,j) '■ 6ij = 1}, where the (5ij's are i.i.d. Bernoulli variable taking value 
one with probability p and zero with probability 1 — p- Because our proofs will heavily rely on 
the dual certificates constructed under the Bernoulli model, we choose this proxy random model to 
the uniform sampling, denoted by ~ Ber{p), in our paper for simplicity. We would like to point 
out that the convenience brought by the Bernoulli model does not weaken the formal theorems; for 
more details see [9]. 

Now we are ready to present two main results for exact matrix completion and robust principle 
component analysis via convex optimization. 
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Theorem 1 ( [TJ Theorem 1.2]). Let Al G ]R"iX"2 q, y^^jg^^ matrix obeying the strong incoherence 
assumption with parameter fi. Let n := max(ni,n2). Suppose we observe m entries of M with 
locations sampled uniformly at random and there is a positive numerical constant C such that 

m > Cfi'^nrlog^ n, (17) 

Then M is the unique solution to convex programming ^ with probability at least 1 — n~^. 

Theorem 2 ( [9l Theorem 1.1]). Suppose M G ]g"ix'^2 Qjj^yg ffi^ incoherence assumption with 
parameter fi, and that the support set of Sq is uniformly distributed among all sets of cardinality 
m. Write n^^i^ := max(ni,n2) and n(2) := min(ni,n2). Provided that 

rank(M) < /9rTi(2)/U~^(logn(i))~^, and m < psnin2- (18) 

where pr and ps are positive numerical constants. Then there is a positive numerical constant c 
with probability at least 1 — cn^^*^ such that {M, Sq) is the unique solution to convex programming 
^ with A = l/^^rT^. 

3 Main results 

In this section, we present the main resuhs of this paper. More precisely, we will show that, if 
r is large enough, then the low rank matrix M can be exactly recovered by the strongly convex 
programming ([5|) for the MC problem and ([6]) for the RPCA problem. A computable and explicit 
lower bound of r will be given. The results are proved in terms of dual certificates (also known 
as Lagrangian multiplier) in combination with a relaxation method introduced in |18j and studied 
in [9]. 

3.1 Matrix completion 

In this section, we discuss the strongly convex programming for the matrix completion problem. 
Recall that, in the MC problem, we would like to recover an unknown low-rank matrix M from the 
sampling of its partial entries V^M. We will show that M is the unique solution of the strongly 
convex minimization ([5]) with dominant probability, provided that r exceeds than a specified finite 
number. We will also give an explicit lower bound of r using only the sampling VnM. 

We first give a sufficient condition for M being the unique solution of ([S]). It follows from the 
standard convex optimization theory and [7]. 

Theorem 3. Assume that there is a matrix Y that satisfies 

(a) Y = VnY, 

(b) V^Y = ^M + UV*, 

(c) \\V^i^Y\\< 1. 

Then M is the unique solution of the strongly convex programming 
Proof. The Lagrangian function associated with the strongly convex programming ([5]) is 

L(X,y) := \\X\U + ^\\X\\l - {Y,VnX -VnM), (19) 



7 



where Y is the the Lagrangian multipher. By the strong convexity of the objective and the first 
order optimahty of the Lagrangian function, M is the unique solution if and only if there is a dual 
matrix Y satisfying 

0£d\\M\U + -M -VnY. (20) 
r 

Since we only use the entries of y on from the expression above, it is more convenient to assume 
Y = VqY. We have known that 9||M||* = UV* + W with VtW = and < 1. Thus the 

sufficient and necessary condition becomes: there exists W such that 

W = Y --M -UV* (21) 
r 

with Y = VnY, VtW = 0, and ||H^|| < 1. Therefore, it suffices to find a matrix Y obeying 

Y = VnY 

Vt{Y - -UV*) = (22) 
\\Vt^±{Y- -UV*)\\ < 1. 

By the expression of Vt and the fact U*U = V*V = Ir, it is not hard to see that Vt{UV*) = UV* 
and VtM = M, and hence Vt^{UV*) = and Vr^±M = 0. So ([22]) is equivalent to 

Y = VnY 

VtY = iAf + UV* (23) 
\\'Pt±Y\\< 1, 

which concludes the proof. □ 

Next, we shall construct a valid dual certificate Y obeying the conditions listed in the theorem 
above. We need the following lemma from |7j. 

Lemma 1 ( [3 Corollary 3.7], [HI Theorem 6]). Under the assumptions in Theorem[l\ we have, 
with probability at least 1 — n~^, 

IIT't^AII < ^, where A := VnVTiVTVnVT)~^UV* , (24) 

and 

p 3^ 

-Z :< VtVuVt ^ "2""^' '"'^^'"^ P ■~ 'ni/{'nin2). (25) 

It is obvious that the matrix A in Lemma [1] satisfies A = "P^A and "PtA = UV* . Now we 
are going to construct a valid dual certificate for exact matrix completion via strongly convex 
programming. 

Theorem 4. Assume 

" 1 - iiptxAii ' 

where A is defined in Lemma [II Then, under the assumptions in Theorem \^ M is the unique 
solution to the strongly convex programming ([5]) with probability at least 1 — . 
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Proof. We prove the theorem by constructing a matrix Y which satisfies conditions in Theorem [3] 
with high probabihty. Indeed, if we define 

y = A + -VnVTiVTVnVTy^ M 

T 

with A defined as in Lemma [H then the conditions in Theorem [3] are satisfied with probabihty at 
least 1 — . 

By Lemma [H with probability at least 1 — n"'^, we have HT^rpxAH < |. We verify (a)(b)(c) in 
Theorem [3] under the events that HP-pxAH < ^. First of all, r > 0. Therefore, dS]) is a strongly 
convex programming. Next, by the construction of Y , it is obvious that (a) and (b) in Theorem [3] 
are true. It remains to show (c). In fact, (j26p implies that 

-WV^i-VnPTiVTVnVTT^MW < 1 - \\Vt^^K\\. 

T 

Since ||'PqxA|| < 1/2, we have 

II'Pt-l^II < II'Pt-lAII + -\\VT±rnVTiVTVnrTy^M\\ < 1. 

T 

Therefore, (c) is verified. □ 

The lower bound of r in Theorem [J] is obviously finite. However, it is hard to estimate in 
practice, as the quantities in ()26p are hard to estimate before we get M. In the following, we 
present a computable lower bound of r, using only the given data VqM. Our starting point is 
Lemma [H which guarantee (j24p and (|25p with very high probability. Notice that 

\\VnVTZ\\j, = {VnVTZ,VnVTZ) = {Z^VtVuVtZ). (27) 

We have 



and 



\\VnVT{VTVnVT)-^M\\l = {{VTVnVT)-^M, M) < -\\M\\l, 

P 



\\VnVTM\\l > |||M|||. 



With these two inequalities, we can easily get 
1 1 Vt^ VnVT {VTVnVT y^M 



l-II^T^AII 



< 2\\V^±VnVT{VTVnVTr^M\\F < 2\\VnVT(VTVnVTy^ M\\f 



< 2.n\M\\F < -\\VnVTM\\F = -WVnMWF. 
\ P P P 



(28) 



This bound can be computed by using only the sampling VqM. We summarize the result into the 
following corollary. 

Corollary 1. Assume 

T>-\\VnM\\F, (29) 
P 

where p = i^^j^ is the sampling ratio. Then, under the assumptions in TheoremUl M is the unique 
solution to the strongly convex programming dSJ with probability at least 1 — n~^. 
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3.2 The Robust Principle Component Analysis problem 

Next, we discuss the RPCA case. Recall that, in the RPCA problem, we would like to recover the 
unknown low-rank matrix M from the sampling D of its complete entries with a small fraction 
of them corrupted. We denote the noise in D by Sq := D — M. Since only a small fraction of 
entries are corrupted, is a sparse matrix. Let be the supported set of Sq. In this section, we 
shall show that (M, Sq) is the unique solution of the strongly convex programming ([6]) with high 
probability, provided r exceeding a finite number. An explicit lower bound of r will be given as 
well. 

Similar to the MC case, we first present a theorem that states a sufficient condition for (M, Sq) 
being the unique solution of ([6]) . 

Theorem 5. Assume {{VuVtW < 1/2. Suppose that there exists a pair (W,F) and a matrix B 
obeying 

UV* + W + -M = A(sgn(5o) + F + VnB) + -5o (30) 
r T 

with 

V^W = 0, IIW^II < /3, V^F = 0, < /3, \\VnB\\F < a, 

where q, /3 are positive parameters satisfying 

/3< 1, 

a + l3<l/\, (31) 
A < (l-/3)/2a. 

Then (M, Sq) is the unique solution of the strongly convex programming 

Proof. The main idea of the proof follows the arguments of [HI Lemma 2.4]. We consider a feasible 
perturbation of the form (M + H,Sq — H) and show that the objective increases unless H = 0. Let 
UV* + Wq + be an arbitrary subgradient of \\L\\^: + ^||L|||, at M, and so we have VtWq = 



|2 

at 5o, and we have VqFq = and ||i*b||oo < 1- By the definition of subgradient, it holds 



and 1 1 Wo 1 1 < 1. Similarly, let A(sgn(5o) + Fq) + be an arbitrary subgradient ofA||5'||i-|-^||5||^ 



f{M + H,SQ-H) 

1 1 (32) 

> /(M, So) + {UV* + Wq + -M, H) - (A(sgn(5o) + i^o) + -^o, H). 



T T 



By the construction in [9], we can always choose Wq in (|32ll such that {Wq,H) = \\Vt^±H\\^:, and 
Fq such that {Fq,H) = — \\'Pq±H\\i. Therefore, 



f{M + H,So-H) 
f{M, Sq) + II^T- 

f{M, Sq) + \\Vt^H\U + \\Vn±H\U (33) 
+ {UV* + -M- Asgn(5o) - -Sq - XVnB, H) + X{VnB, H) 

T T 

f{M, Sq) + \\Vt±H\\, + \\V^±H\\i + {-W + AF, H) + X{VnB, H), 



> f{M, Sq) + \\V^±H\U + \\Vn±H\\i + {UV* + -M- Asgn(5o) - -Sq,H) 

T T 
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where in the last equahty we have used the assumption (j30p . By the triangle inequality and the 
assumptions on W and F, we derive that 



-W + XF,H)\ < \{W,H)\+X\{F,H)\ 

= \{W,V^^H)\+X\{F,Vn±H)\ 
<\\W\\-\\Vr^^H\U + X\\F\\^-\\V^^H\\i 
<m'PT^H\\* + ^'P^^H\\i)- 



(34) 



By the Cauchy-Schwarz inequality, it holds 

\{VnB,H)\ = \{VnB,VnH)\ < a\\VnH\\F. (35) 

Furthermore, we have 



\\VnH\\F < WVnVTHWF + \\rnVT^H\\F < ^\\H\\f + ^t^^Hf 
< ^\\VnH\\F + ^WVn^HWF + \\Vt^H\\f, 



(36) 



which implies 

\\VnH\\F < \\Vn±H\\F + 2\\V^^H\\f < \\Vn±H\\i + 2\\V^^H\U. (37) 
Combining all the inequalities above together, we get 



f{M + H,So-H)-f{M,So) 

>(1 - /5 - 2aX)\\V^±H\U + (1 - A/3 - Xa)\\V^±H\\i. 



(38) 



This, together with ([3T]) . implies that (M, Sq) is a solution to ([6]). The uniqueness follows from the 
strong convexity of the objective in ([6]). □ 

Before we derive a lower bound of r for the RPCA problem, we need the following lemmas. 

Lemma 2 ( [9l Lemmas 2.8]). Assume that Q Ber{p) with parameter p < ps for some ps > 0. 
Then there exists a matrix in the form ofW^ = V'^±Y satisfying: under the other assumptions 
of Theorem\^ 

(a) \\W^\\ < 1/4, 

(b) \\Vn{UV* + W^)\\F<X/4, 

(c) \\V^^{UV* + W'^)U<X/A. 

Lemma 3 ( [HI Lemmas 2.9]). Assume that Sq is supported on a set VL sampled as in Lemma\^ 
and that the signs of Sq are i.i.d. symmetric (and independent ofQ). Then there exists a matrix 
in the form ofW'^ = V'^^Z satisfying: under the other assumptions of Theorem\^ 

(a) \\W^\\ < 1/4, 

(b) WVn^W^Wo. < A/4, 

The construction of and can be found in [9]. Now, we show that (M, 5*0) is the unique 
solution to the strongly convex programming ^ with high probability. The idea is to verify that 
is a valid dual certificate when the parameter r goes beyond some finite value. More 
precisely, we check the conditions in Theorem [5j 
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Theorem 6. Assume 

Then, under the assumptions of Theorem\^ (M, 5o) is the unique solution to the strongly convex 
programming ([6]) with probability at least 1 — cn'^^^^ , where c is the numerical constant in Theorem 

m 

Proof. We prove the theorem by checking the conditions in Theorem [5l Due to the relation between 
F, W and B in (j30p . it suffices to show that, there exists a matrix W obeying 

VtW = 0, 

\\W\\<f3, 

\\Vn4UV* + W + lM-^So)\\oo<l3X, ^^^^ 
\\Vn{UV* + W- Asgn(5o) + \M- \So)\\f < aX, 

where a, (3 satisfies (f3T]) . 

Let W = + with and W'^ defined in Lemmas [2] and [3] respectively. We show that 
(j40p holds true. For simplicity, we denote 

7:= \\Vn±{M - So)\\oo, S:= \\Vn{M - So)\\f. 

We choose a = ^ and /? = 1 — e, where < e < ^ is a parameter determined later. Then, 
([HT|) is satisfied for any A < 1; note that the final choice A = l/y^n(Yy < 1. By the expressions 
of and W^, it is obvious that W £ T^. Since HT^-^H < 1/4 and \\W^\\ < 1/4, we have 
IIM^II < \\W^\\ + \\W^\\ < 1/2 < /3. For the third inequality in (jiO]), we have 

\\Vn^{UV* + W + ^M-lSo)\\oo 

< WVn^iUV* + W^)\\^ + WVn^W'^Woo + IWPn^iM - 5o)||oo 

< | + | + ^ro4M-5o)||oo ^ > 

For the last inequality in ()40p . we notice that 'Pc(^'^) = Asgn(S'o) as shown in [9], and therefore 
we have 

\\Vn{UV* + W- Asgn(5o) + - ^So)\\f 
= \\Vn{UV* + W -W^ + ^M-^So)\\f 

= \\Vn{UV* + + -^So)\\f (42) 
< \\Vn{UV* + W^)\\f + \\\Vn{M - Sq))\\f 

— A _L i 

In order that the last two inequalities in ()40p are satisfied, we have to choose a r such that 

^ + ^<(1-^)A, and ^ + ^<1 
It 4 r z 



which implies that 

T > max < 

(i-6)A'(5 




r > max<^ a ^ W 1 e " x-, ) ■ (^3) 
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To get a tighter lower bound of r, we choose e to minimize the maximum in the right hand side of 
(I43p . Since ,i monotonically increases and '^^^ monotonicahy decreases as e increases from 

4 to i , it is easy to see that the maximum is attained when ,-, = ,,^k, , which yields a solution 

2 2' ^ (i-f)A (f-T) 

i I 7 



e = that always falls in ^] if A < 1. Substituting it into (j43]) . we obtain an optimal lower 
bound of T 



^ 27 + A5 
r > 



A(l-A)- 

In viewing of 'Pq±So = and therefore Vq±{M — Sq) = Vq±D, we get (139]) . 

In the above derivation, we have used only assertions (a)(b)(c) in Lemma [2] and (a)(b) in 
Lemma [3l The probabilities that these assertions hold true and IIPhPtH < 1/2 are shown in the 
proof of [HI Theorem 1.1], under the same assumptions of this theorem. Therefore, the remaining 
of the proof of this theorem follows directly from that of [9l Theorem 1.1]. 

□ 

It is easy to see that the lower bound of r is a finite number. However, the exact lower bound 
is very hard to get, because we can only manipulate the given data matrix D. In the following, we 
give a method to estimate the lower bound. For this, we have to get upper bounds for the norms 
involved in (j39p . For the first norm, we simply use 



WPn^DW^ < Halloo- 

For the second norm, by using the facts V^±M = Vfi±D and ||'Pn'PT|| < 1/2, we have 
WVnMWl = ^f^^TMllI < i||M||| = ^(llPnMllI + \\Vn^D\\l), 

which implies 

\\rnM\\p<^\\rn±D\\F. 

Therefore, we obtain 

WVniM - So)\\f = pVuM - VnDWp < WVuDy + 2\\rnM\\F < WVnDWp + ^\\Vn±D\\F. 
By using the Cauchy-Schwarz inequality, we finally have 

\\Vn{M-So)\\F< 



\ 




Therefore, in practice, we can choose 



2||L»||oo + ^l|I)| 



A(l-A) 

to guarantee the exact sparse low-rank matrix decomposition by solving the strongly convex opti- 
mization ([6|). Similar to Corollary [1] for exact matrix completion, we have an analog lower bound 
estimate based on the observed data matrix for exact low-rank and sparse matrices decomposition. 
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Corollary 2. Assume 

^ 2\\D\\^ + ^\\D\\f 

y _ii II J II ii_ 

A(l-A) 

Then, under the assumptions of Theorem\^ {M,Sq) is the unique solution to the strongly convex 
programming with probability at least 1 — cnj"^^", where c is the numerical constant in Theorem 

m 

4 Conclusion 

In this paper, we have shown that strongly convex optimizations can lead to exact low-rank matrix 
recovery in both the matrix completion problem and robust principle component analysis under 
suitable conditions. Explicit lower bounds for the parameters r involved are given. These results 
are complementary to the results in [6l[71[9], where convex optimizations are shown to lead to the 
exact low-rank matrix recovery. 

We would like to point out that the combination of the MC and RPCA problems, i.e., matrix 
completion from grossly corrupted data [9] has been modeled as 

minimize: ||L||* -|- AH^Hi 
subject to : V^{L + S) = Y. (44) 

With the technique discussed in this paper, one can modify ()44p to its strongly convex counterpart 
that can guarantee the exact recovery as well. 

Note that all models here assume that the observed data are exact. However, in any real world 
application, one only have observed data corrupted at least by a small amount of noise. Therefore, 
a possible direction for further study is to consider the case of noisy data [8] in matrix completion 
and robust principle component analysis. 
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